Self-organization in populations of competing agents 
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A population of heterogenous agents compeeting through a minority rule is investigated. Agents 
which frequently loose are selected for evolution by changing their strategies. The stationary compo- 
sition of the population resulting for this self-organization process is computed analytically. Results 
are compared with numerical simulations of two different minority games and other analytical treat- 
ments available in the literature. 
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The mechanisms under which a population of compeet- 
ing agents self-organizes is a problem which has gained a 
lot of imterest among the physics community in the last 
few years 0-0]. As in traditional statistical mechanism 
the main goal is to determine the average behavior of the 
population based on the elementary rules which charac- 
terize the competition among agents. Given the actual 
state of art most of the works have been devoted to the 
characterization, by means of numerical experiments or 
analytical treatments, of the different models proposed in 
the literature 0-00], while a general description is still 
missed. 

This work is an attempt to extract some general fea- 
tures present in the subclass of models where the perfor- 
mance of the agents is characterized by certain minority 
rule 0,0,0 . Agents are assumed to be heterogeneous and 
the main goal is to obtain which is the final composition 
of the population if certain evolution mechanism is intro- 
duced. The agents will be supposed not to be subscribed 
to any regular lattice which rule out the existence of spa- 
tial correlations allowing a mean field (MF) treatment. 

The work is organized as follows. First the general 
features of the class of models under consideration are 
introduced. Based on these rules the MF rate equations 
describing the evolution of the population are derived and 
the stationary solutions are computed. These results are 
compared with numerical simulations of two particular 
minority "games". Moreover, a comparison with the an- 
alytical results already available for the model by John- 
son et al 0-0| is also drown. In all cases a very good 
agreement is obtained. 

Considers a population of N (odd) heterogeneous 
Boolean agents characterized by certain property x. The 
Boolean nature of the agents restrict their action to only 
two possibilities. An example is the Arthur-bar problem 
0| in which each agent attend or not a bar based on the 
past attendance to the bar. Another example is the mi- 
nority game introduced by Challet and Zhang 0] in which 
the agents can either buy or sell. In both models a good 
choice of x is the probability that an agent takes differ- 
ent decisions in two different steps. A different choice is 
taken in the model introduced by Johnson et al 0]. In 
this case x is the probability that an agent accepts the 



decision suggested by its strategy or does the opposite. 
In general one make think in x as any property, or set of 
properties, which distinguish the different strategies one 
agent can choose. 

On each step each agent take one of the two decisions 
and those being in the minority win. The long time per- 
formance of the agents will be characterized by a cumula- 
tive index z in such a way that each time an agent win it 
is rewarded with a positive point (z — > z + 1), otherwise it 
is punished receiving a negative point (z — > z — 1). Evo- 
lution in the population is introduced assuming that each 
time the performance z of an agent goes below a thresh- 
old — z c (z c > 0) it is selected for evolution an change its 
strategy: a new value of x (x — > x') is selected among 
certain distribution Pq(x') and its performance is reseted 
to zero (z — ► 0). 

A similar evolutionary mechanism has been already 
considered in [0. One also may think in other imple- 
mentations like the extremal rules considered in [00] , or 
a probabilistic rule in which agents can change strategy 
even below z c according to certain probability, which in 
general may depend on z. In any case it is expected that 
all of them give the same qualitative behavior in the large 
z c limit. 

Having defined the model let us determine which is the 
stationary composition of the population of agents P(x), 
the fraction of agents of type x. For doing so we first 
derive the rate equations which describe the dynamical 
evolution of P(x), which in general can be written as 
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Y,[W{x',x)P{x') 



W(x,x')P(x)], (1) 



where W(x',x) is the transition rate from x' to x. 

The only rule which allows a change on x is the evo- 
lution rule, in which the agent chose a new value of 
x' , selected with probability Po(x'). Thus, if X(x) is 
the fraction of :r-agents which are changing their strat- 
egy per unit time then the transition rates are given by 
W(x,x') — X(x)Po(x'). After substitution of this expres- 
sion into eq. ([l]) it results that 
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P(x) = P (x) J2 A(x')P(x') - \{x)P(x). 



(2) 
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To determine X(x) we need to consider the temporal 
evolution of the cumulative performance index z(x). Let 
P m (x) be the probability per unit time that a x-agent 
is in the minority in one step. Thus, which probability 
P m {x) (1 — P m (x)) the performance increases (decreases) 
by one unit. More generaly one can define P m (x) as the 
probability that an agent wins a point, regardless how 
it did. On the other hand, if an agent changes strat- 
egy, which happens with probability A (a;), then its per- 
formance increases by z c . These elementary processes 
lead to the rate equation 
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z{x) = 2P m (x) -l + z c X(x). 



(3) 



Provided P m {x) < 1/2 for all x the system described 
by eqs. (||) and eq. (^) will always reach a stationary 
state. In this state time derivatives vanish obtaining 



A(a;) = -[l-2P m (x)l 



P(x) = 



1 - 2P m {x) 



Po(x), 



(4) 



(5) 



where A is a normalization constant. 

Agents with P m (x) close to 1/2 are more stable to 
change in strategy as it can be seen from eq. (^) and, 
therefore, they will have a larger participation ratio in 
the stationary population, as follows from eq. (||). The 
distribution from which the value of x is extracted is thus 
modulated by the probability of being in the minority, 
yielding the stationary population in eq. (Q). 

On the other hand, from eq. (|j) it can be seen that in 
the limit z c — > oo the fraction of agents changing strategy 
becomes infinitesimal and, therefore, in this limit one ex- 
pect to obtain the same results as if one use an extremal 
evolution rule. Another aspect to be emphasized is the 
fact that the parameter z c , which in principle is the only 
evidence of the particular evolution rule chosen here, does 
not appears in the expression for the stationary distribu- 
tion of agents in eq. (||) and, therefore, this result is 
expected to hold independent of the particular evolution 
rule chosen. 

So far the results obtained here are very general and 
expected to apply to a wide class of models of Boolean 
agents with minority rule. To go further we have to de- 
termine P m (x) which may however depends on the par- 
ticular model under consideration. Below two different 
cases are analyzed. One is a very simple model of non- 
interacting agents (NIA) which make their decision at 
random. The other is the very recent implementation of a 
population of Boolean agents build up onto a Kauffman's 
network (AKN) introduced by Paczuski and Bassler J?]]. 

In the AKN model introduced in J?J agents makes their 
decisions based on the previous decision of some other 
agents. A Boolean variable <x; (i = 1, 2, . . . , N; ai = 0, 1) 
is assigned to each agent which is representative of the 



two possible decisions one agent may take. The decision 
taken by each agent is based on the decision took by other 
K agents chosen at random . . . ,1k) m the previous 
step, according to certain Boolean function fi selected at 
random among the set of all the 2 2 possible Boolean 
functions with K inputs, i.e. 



0-i(£ + l) ^ fi[<7 n {t),...,a iK {t)}. 



(G) 



Without lost of generality and for a reason which be- 
comes clear below, Booleans functions which give the 
same output independent of the inputs are rule out. 

In the original variant of this model J?J an extremal 
evolution rule is used, such that after certain time the 
worst agent is selected for evolution. This rule is relaxed 
here with the barrier evolution rule considered above, 
which yields the extremal dynamics as z c — > oo. More- 
over, one can see that a property that makes differences 
among the agents can be the number of times 1 appears 
on the M = 2 K outputs of its Boolean function, denoted 
by n. If n is close to M/2 the agent will give as output 
or 1, depending on the configuration of its K neighbors, 
with approximately the same probability. Otherwise, if n 
is close to 1 or M the agent will practically give the same 
output independent of the configuration of its neighbors. 

The NIA model is a simplification of the above model 
in which interactions among agents are rule out. In this 
case on each step an n-agent gives 1 as output with 
a probability n/M and otherwise. For this case the 
agent's decision does not depends on decisions of any 
other agents and, therefore, there is no other correlation 
than the one introduced by the minority rule which de- 
pends on the output of all agents. 

Numerical simulations were performed for a popula- 
tion of N = 99 agents and K = 2,3, 4, 5. In all cases the 
system is updated until it reaches the stationary state 
and then average is taken over the temporal evolution 
of the population. The resulting data is averaged over 
different realizations of the initial Boolean functions and 
over the choice of neighbors in the interacting case. 

Since the new Boolean functions are selected at ran- 
dom one has 



M-l 

P (n) = B(n; 0.5, M)/ ^ B{r; 0.5, M), 

r=l 



(7) 



where B(n;p,m) is the Binomial distribution, the prob- 
ability to obtain 1 n times and m — n times given on 
each single event 1 happens with probability p. The val- 
ues n — and M are rule out because they give fixed 
strategies and consequently the Binomial distribution is 
renormalized. 

To compute the stationary composition P(n) one has 
to compute the probability P m (n) that an agent of type 
n is in the minority, and then plug in the result in eq. 
(||). For N large the game is expected to be symmetric 
in the sense that with probability 1/2 the minority is the 
group of agents which takes the 1 (or 0) as output. In 
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such el CcLS6 cL fixed agent will, in a long time window, have 
a probability 1/2 of being in the minority while agents 
changing their output very often are expected to has a 
lower probability to be in the minority. 

The main hypothesis taken here is that 1 — 2P m (n) = 
f[p(n)], where f[p(n)} is a smooth function of the proba- 
bility per unit time p{n) that an agent change its output. 
Since fixed players (p — 0) has a probability 1/2 to be 
in the minority then /(0) = 0. In the following f(p) is 
expanded around p — 0, keeping only the linear term, 
resulting 



1 - 2P m (n) cx p(n) + 0[p{nf 



(8) 



Now, for NIA p(n) is just the probability to find two 
different outputs on the agent's strategy, which is given 
by P c (n) = 2(ra/M)(l - n/M). For AKN one should also 
take into account that agents can changes their output 
only if at least one of its inputs has changed output in 
the previous step. Within a MF approximation the last 
event happens with probability Kp. Thus, in general 



p(n) = BP c (n), 



(9) 



where B = 1 and B = Kp for NIA and AKN, respec- 
tively. 

By construction in both models P c (n) > because 
the cases n = and M have been rule out. Hence, 
1 — 2P m (n) cx BP c (n) can only be zero if B = and 
if does it is zero for all n. If the second possibility hap- 
pens then all agents will have the same probability to 
be in the minority and, therefore, their distribution in 
the stationary state will be the distribution from where 
the Boolean functions are extracted, i.e. P(n) = Po(n). 
Hence, eqs. (^),(||) and(^J) yields the following alternative 



P(n) = <j PcM 
^ Po(n 



P (n) forB > 
) forB = 0, 



(10) 



where P c = P c (n)P(n), which is the final output of 
the present calculation. 

For NIA as mentioned above B = 1 and, therefore, the 
alternative B > takes place. The comparison of this 
prediction with numerical data is shown in fig. |l|. The 
agreement is quite well for all values of K, proving that 
the ansatz in eq. ([)]) applies for this model. 

For AKN B = Kp and one should analyze weather p 
is zero or not. As it is well known the Kauffman's net- 
work display qualitative different behavior depending on 
the value of K ||. For K < 2 and independent of the 
initial conditions the network evolve to a frozen config- 
uration with p = 0. On the contrary for K > 2 the 
network evolves to periodic orbits with period growing 
exponentially with N ("chaotic phase"), in which p > 0. 

Thus, the population of agents build up onto the Kauff- 
man's network is a very good scenario to test the validity 
of eq. ( |l0| ) because both alternatives can be observed. 
The comparison is shown in fig. [I]. For K = 2 it can 
seen that the numerical data is in better agreement by 



P(n) — Po(n) as predicted above. On the contrary for 
K > 2 the data is in better agreement with the case 
B > in eq. @. 

For K > 2 both the NIA and AKN models yields the 
same distribution P(n) and, therefore, the correlations 
introduced by the network are in those cases irrelevant. 
Moreover with increasing K the composition of the pop- 
ulation gradually approaches Pq(ji) which explains the 
lost of self-organization observed in fTfl. 

Finally the model of Johnson et al^^ is considered. In 
this case the decisions taken by the agents depends on the 
previous history of the winning group. For each agent it is 
available the information of which has been the winning 
group a w (t) (a w (t) — 0,1) in the last K steps. More- 
over, to each of then and strategy /^is assigned, which 
is extracted at random among all the possible Boolean 
functions of K inputs. Then, for each possible past his- 
tory each agent will give a well defined output based on 
its own strategy, i.e. 



o-i(t + 1) = fi[a W (t), <Jw(t -K+l)]. 



(11) 



Notices that in this case the output of each agent de- 
pends on a global information and not on the outputs of 
any other agent as in the AKN model (see eq. @). 

So far this model is mst a variant of the minority game 
of Challet and Zhang j^] for the case in which each agent 
has only one strategy at his disposal. In the variation by 
Johnson et al || a probability pi is assigned to each agent 
based on which the he accepts or not the output of its 
strategy. With probability pi (1—pi) he use the outcome 
(the opposite outcome) of its strategy. Moreover, they in- 
troduced the cumulative index Zi to measure the perfor- 
mance of each agent in a similar way as described above. 
The only difference is that for this model when zi goes 
below the threshold — z c the agent does not change its 
Boolean function fi but rather its probability pi, choos- 
ing a new one at random in the interval [pi— R/2,pi+R/2] 
with reflective boundary conditions. 

Some analytical results are already available for this 
model [0,!J ■ U sm S a diffusion like approach Q or a de- 
tailed probabilistic calculation [|| it has been shown that 
in the stationary population has the following composi- 
tion 



P(p) 



A 



i-2P m { P y 



(12) 



where A is a normalization constant and P m (p) is the 
probability that an agent of type p is in the minority [^) . 

Eq. dl2| ) is actually quite similar to eq. (||), with the 
choice x = p. Since for this model the new values of p 
are extracted from a uniform distribution it is expected 
that Pq(p) does not depend on p. Hence, eq. ( |l2| ) can 
be seen as a limiting case of eq. (||) when applied to the 
particular evolution rule of the model by Johnson et al 
H where Pq{x) = const.. 

In order to go beyond this result one has to deter- 
mine P m (p)- This has already been done in resulting 



3 



that for large iV 1 - 2P m (p) w C(N)2p(l - p) where 
C{N) ~ N^ 1 / 2 . This results does not seems to have 
any relation with the ansatz in eq. (^p[). However, one 
should notice that 2p(l — p) is just the probability P c (p) 
that an agent of type p give two different outputs, given 
the output of its Boolean function has remained fixed. 
Therefore, from this result and eq. (Q2h it follows that 



P(p) = 



P 



Pcip) 



(13) 



This eq. is just the B > alternative of eq. (|l0|). Hence, 
the functional dependence of P m (x) and P(x) on P c {x) 
appears to be universal for the class of models studied 
here. 

In summary, a heterogenous populations of compeet- 
ing agents has been studied. Based on general arguments, 
such as the minority rule and evolution, the stationary 
composition of the population of agents has been com- 
puted as a function of the probability P m {x) of being in 
the minority. Further analysis reveals that the relation 
1 — 2P m (x) oc P c (x) is universal for the class of model, 
where P c (x) is the probability that an agent of type x 
gives different outputs. 

For the AKN it is concluded that except for K = 2 
the correlations introduced by the network are irrelevant 
and agents can be considered to be independent. In the 
particular case K = 2, which correspond to the critical 
network, the population build-up onto the Kauffman's 
network reach an stationary state in which all agents have 
a probability 1/2 to be or not in the minority. Moreover, 
the lost of self-organization with increasing K was shown 
to takes place gradually. 
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FIG. 1. Stationary composition of the population of agents. 
The points where obtained from numerical simulations of a 
population of N — 99 agents with a threshold z c = 100 and 
K = 2 (squares), K — 3 (circles) and K = 4 (triangles). In 
all cases the open and full symbols corresponds to the NIA 
and AKN models, respectively. The solid lines are obtained 
using eq. (0) with B = (dashed lines) and B > (solid 
lines) for the corresponding values of K. 



4 



